Television related searching

ABSTRACT

The subject matter of this specification can be implemented in, among other things, a computer-implemented method that includes identifying metadata related to television programming being presented on a display device. The method further includes extracting one or more keywords from the metadata. The method further includes generating multiple search suggestions based on the keywords and first search results based on one or more of the search suggestions. The method further includes presenting the search suggestions and the first search results together on the display device.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims the benefit under 35 U.S.C. §119(e) of U.S.Patent Application No. 61/346,457, entitled “Television RelatedSearching”, filed May 19, 2010, which is incorporated by referenceherein in its entirety.

TECHNICAL FIELD

This instant specification relates to television related searching.

BACKGROUND

Many people like to multi-task while watching television. In some cases,a person may like to surf the web or chat with friends while watchingtelevision. In some cases, a person may play a game while watchingtelevision. These activities sometimes draw viewers away from televisionto their personal computer, cell phone, game console, or other networkconnected device.

Sometimes, computer users perform searches for content that is availableon the Internet. For example, a user may browse to a web site for asearch engine. At the search engine web site, the user enters a searchquery. The search query generally includes words, phrases, or operatorsthat describe what the user is looking for. The search engine processesthe search query and returns a set of search results to the user. Thesearch results generally include links to web pages or other contentavailable on the Internet that match one or more terms in the searchquery. The user can then select a link in the search results to navigateto the corresponding web page.

Some enhanced television systems, such as WINK, OPENTV, ADVANCEDTELEVISION ENHANCEMENT FORUM (ATVEF), YAHOO WIDGETS, and OPENCABLEAPPLICATION PLATFORM (OCAP), can display Internet content on top oftelevision content. In some cases these systems use triggers built intothe audio/video signal to link particular television shows oradvertisements with a specific interactive application. These systemscan associate an application with a particular point of a particularprogram.

SUMMARY

In general, this document describes television related searchsuggestions. In one aspect, a computer-implemented method for televisionrelated searching includes identifying metadata related to televisionprogramming being presented on a display device. The method furtherincludes extracting one or more keywords from the metadata. The methodfurther includes generating multiple search suggestions based on thekeywords and first search results based on one or more of the searchsuggestions. The method further includes presenting the searchsuggestions and the first search results together on the display device.

Implementations can include any, all, or none of the following features.The method including receiving a user selection of at least one of thepresented search suggestions; generating second search results based onthe selected search suggestion; and presenting the second search resultson the display device. The search suggestions are automaticallygenerated upon identifying the metadata or extracting the keywords.Identifying the metadata or extracting the keywords includes determiningthat the metadata or the keywords identify a location of additionalcontent. The location identifies a television program, a televisionchannel, a web page, or an application. Generating the first searchresults includes sending a search query that includes the one or moresearch suggestions to a search engine system. Generating the firstsearch results includes searching a local data storage. The methodincluding monitoring a channel of the television programming beingpresented, and wherein identifying the metadata includes retrievingelectronic program guide data for the channel, current day, currenttime, and head end system of the television programming being presented.Identifying the metadata includes recognizing text in an audio or videosignal of the television programming. The first search results includean identifier of a television program currently being broadcast, atelevision channel, a web page, or an application; the method furtherincluding receiving a user selection of the identifier and, in response,causing content for the selected identifier to be presented on thedisplay device. The first search results include an identifier of atelevision program scheduled to be broadcast at a future time; themethod further including receiving a user selection of the identifierand, in response, causing content for the selected identifier to berecorded at the future time.

In one aspect, a computer-implemented method for television relatedsearching includes outputting television programming for presentation ona display device. The method further includes presenting a text inputcontrol on the display device. The method further includes receiving atleast one user input in the text input control. The method furtherincludes generating multiple search suggestions based on the user inputand first search results based on one or more of the search suggestions.The method further includes presenting the search suggestions and thefirst search results together on the display device.

Implementations can include any, all, or none of the following features.The method including receiving a user selection of at least one of thepresented search suggestions; generating second search results based onthe selected search suggestion; and presenting the second search resultson the display device. Generating the first search results includessending a search query that includes the one or more search suggestionsto a search engine system. Generating the first search results includessearching a local data storage. The first search results include anidentifier of a television program currently being broadcast, atelevision channel, a web page, or an application; the method furtherincluding receiving a user selection of the identifier and, in response,causing content for the selected identifier to be presented on thedisplay device. The first search results include an identifier of atelevision program scheduled to be broadcast at a future time; themethod further including receiving a user selection of the identifierand, in response, causing content for the selected identifier to berecorded at the future time.

The details of one or more implementations are set forth in theaccompanying drawings and the description below. Other features andadvantages will be apparent from the description and drawings, and fromthe claims.

DESCRIPTION OF DRAWINGS

FIG. 1 is a schematic diagram that shows an example of a system fortelevision related searching.

FIG. 2 is a schematic diagram that shows an example of a system fortelevision related searching.

FIG. 3 is a block diagram that shows an example of a search system fortelevision related searching.

FIG. 4 is an example of a graphical user interface for televisionrelated searching.

FIG. 5 is a flow chart that shows an example of a process for televisionrelated searching.

FIG. 6 is a schematic diagram that shows an example of a computingsystem.

DETAILED DESCRIPTION

FIG. 1 is a schematic diagram that shows an example of a system 100 fortelevision related searching. The system 100 is used to presenttelevision programming and additional content available from theInternet to a user. In some implementations, the additional content isrelated to the television programming currently being presented to theuser. The additional content can include, for example, web pages,gadgets, and/or applications. Gadgets are small interactive applicationswritten in a markup language, such as Hypertext Markup Language (HTML)or Extensible Hypertext Markup Language (XHTML). Gadgets are sometimesalso referred to as widgets. In some implementations, a gadget is aniframe HTML element embedded within a container document or web page. Inanother implementation, a gadget can be a standalone web page.Applications are often written using a programming language, such asJAVA or OBJECTIVE C. The written programming is then compiled into anapplication by a compiler, either sometime prior to running theapplication, at runtime, or some combination of before runtime and atruntime. Often, the written programming is first compiled into bytecode,which is hardware independent. The bytecode may then be run by a virtualmachine, such as the JAVA VIRTUAL MACHINE or the DALVIK VIRTUAL MACHINE.

The visual and audible output of the additional content can be shownsuperimposed on the television programming or, in the case of video,around the television programming. For example, visual output can beshown in the blank space that exists where the aspect ratio oftelevision programming content does not match the aspect ratio of thedisplay device presenting the television content. Alternatively, thedisplay device can be split, for example, into two halves or fourquarters, where the television programming and the additional contentare presented in separate portions. In another example, the televisionprogramming can be presented in a thumbnail or in a corner of thedisplay device on top of the additional content that is presented fullscreen or vice versa.

In some implementations, the additional content is presented on adisplay device that is separate from the device that presents thetelevision programming. Whether on the same display device or a separatedisplay device, the additional content is presented to the user whilethe television programming content is presented to the user. Theadditional content can provide a user with functionality related to thetelevision content the user is currently viewing, such as chatting withother viewers, access to fan websites, or targeted advertising.

The system 100 includes a television programming display device 102, atleast one television programming provider 104, and a search device 106.The television programming display device 102 is an electronic deviceused to display television programming. For example, the televisionprogramming display device 102 can be a television, computer monitor andspeakers, portable video player, or smart phone.

The television programming display device 102 presents televisionprogramming provided by the television programming provider 104. Thetelevision programming can include, for example, television shows,movies, and/or commercials. The television programming provider 104sends the television programming to the television programming displaydevice 102 over a wire line connection, such as cable television ortelevision provided by a telephone company, or a wireless connection,such as broadcast television or satellite television.

The search device 106 monitors the television content being displayed bythe television programming display device 102 and also displays theadditional content on the television programming display device 102. Insome implementations, the additional content is related to thetelevision programming currently being displayed to the user. In someimplementations, the search device 106 controls the tuning of televisionchannels and/or playback of other television programming. For example,the user can send a command to the search device 106 using a wire lineor wireless input device, or a button directly on the search device 106.The command can request presentation of a specific channel, navigatingup or down a channel, a particular video on demand (VOD), or playback ofrecorded programming. Wireless input devices can include, for example,an infrared (IR) remote control or BLUETOOTH and WIFI wireless devices,such as a smart phone, mobile computer, or a desktop computer. Wire lineinput devices can include, for example, a computing device connected tothe search device 106 by a local area network (LAN) or the Internet. Thesearch device 106 receives the request to change the channel and tunesin the requested channel or retrieves the requested video on demand orrecorded programming.

Alternatively, the search device 106 can receive the channel changecommand and pass the channel change command on to an external tuner,such as through an HDMI connection to a tuner in a cable televisionset-top box 105 a or a satellite television set-top box 105 b. Inanother example, the search device 106 can send commands to an externaltuner using a wireless connection, such as IR. In another example, thesearch device 106 and one or more of the set-top boxes 105 a-b cancommunicate using a network, such as a local area network or theInternet. The search device 106 then receives the video and/or audio forthe tuned-in channel from the external tuner.

In another external tuner example, the channel change commands may besent directly to the external tuner or other external programmingdevice, such as a digital video recorder (DVR). The search device 106may then receive the tuned-in channel or other audio/video signal fromthe external tuner or programming device. The search device 106subsequently processes the received programming to identify themetadata. In some implementations, the search device 106 can monitor awire line or wireless connection to the external tuner/programmingdevice to identify programming change commands sent to the externaltuner/programming device. For example, the search device 106 can includean IR receiver for monitoring IR commands sent to an external device.

Whether by an internal tuner, an external tuner, programming on demand,or recorded programming, the search device 106 can decode or identifymetadata for the programming. The metadata is used by the system 100 toselect additional content related to the television programming beingpresented. The metadata may be in-band with the audio/video signal orout-of-band. For example, the search device 106 can extract in-bandclosed captioning text, station identification information, ortelevision program identification information from the audio/video. Inanother example, metadata may be stored out of band with recordedprogramming, such as a file name or file attributes. In another out ofband example, the search device 106 can retrieve electronic programguide (EPG) data or a transcript for a particular show, channel, and/ortime slot from an internal storage at the search device 106 or anexternal storage, such as the set-top boxes 105 a-b or a computingdevice on the same network as the search device 106 or on the Internet.

In yet another example of identifying metadata, the search device 106 oranother system can perform speech recognition on the audio to identifytext or image recognition on the video to recognize text or objects. Forexample, a system in communication with the search device 106 can storeidentifying information for television programming, such as a name,channel, broadcast date/time, or a portion of the audio/video. Thesystem can also store pre-recognized transcripts of audio and/orpre-recognized text or objects for television programming. The systemcan compare the stored identifying information to correspondinginformation for the television programming being presented to the userto determine what television programming the user is currently watching.The system can then retrieve a pre-recognized transcript of the audioand/or pre-recognized text or objects from the video for the identifiedtelevision programming.

The search device 106 combines an audio/video representation of theadditional content with the audio/video signal for the tuned-in channel.For example, the search device 106 can overlay video for the selectedadditional content on top of or adjacent to the existing video for thetuned-in channel. In some implementations, the search device 106 uses asplit screen to display the existing video on one side of the split andthe additional content on the other. In some implementations, the searchdevice 106 overlays audio for the additional content onto the existingaudio. The search device 106 then passes the altered audio/video on tothe television programming display device 102 for presentation to theuser.

In the examples above, the search device 106 is a stand-alone electronicdevice connected to the television programming display device 102 (andin some cases an external tuner). However, in some implementations, thesearch device 106 can be a component incorporated within a televisionprogramming display device or an external tuner. In someimplementations, whether the search device 106 is a stand-alone deviceor integrated within another device, as the television programming beingdisplayed to the user changes, the search device 106 updates theadditional content presented to the user.

Television programming changes can include, for example, a change in thechannel, a change in the television program (e.g., the channel mayremain the same, but a new television program may be presented at aparticular time of day), or a change within a television program (e.g.,a new guest appears on stage during a talk show television program).Updates to the additional content being presented can include, forexample, presenting a new gadget, web page, list of search resultsand/or suggestions, or application. Updates can also include no longerpresenting a previously presented gadget, web page, list of searchresults and/or suggestions, or application. In another example, updatescan include changing the information within or provided to a gadget, webpage, list of search results and/or suggestions, or application.

In some implementations, the search device 106 automatically creates asearch query 110 using the metadata from the in band and/or out of bandsources. The metadata is based on the television content being displayedon the television programming display device 102. The search query 110can include information about the television programming, such as actorsand actresses shown, programming genre, closed captioning text and/orother information. Alternatively, the search device 106 can receive thesearch query 110 as an input from a user. For example, the user mayenter the search query 110 using a remote control or keyboard that is incommunication with the search device 106. In another implementation, theuser may make an input requesting that a search be performed withoutinputting the words or search terms to be used in the search query 110.The search device 106 may then use the metadata to generate the query.

The search device 106 sends the search query 110 to a search system 108through a network 118, such as the Internet. In some implementations,the search device 106 can send the metadata to the search system 108.The search system 108 can then create the search query 110. In someimplementations, the search system 108 creates and sends multipleproposed search queries to the search device 106. The search device 106presents the proposed search queries to the user and the user can selecta search query to perform. The search device 106 then sends the selectedsearch query to the search system 108.

In response to receiving the search query 110, the search system 108performs a search of information, such as web pages, gadgets,applications, or other files, accessible over the network 118. In someimplementations, the information through which the search system 108searches can include electronic program guide information for televisionprogramming accessible by the search device 106 and/or the set-top boxes105 a-b. In some implementations, the search system 108 provides theresults of the search to the search device 106 as search results 112.The search results 112 can include, for example, a list of televisionprograms, channels, web pages, Internet video, or applications havingattributes that match terms in the search query 110. The search results112 can include a reference to a web page or Internet video stored at anetwork application data provider 114. The search system 108 can orderthe items in the list of the search results 112 based users' ratings ofthe individual items, preferences of the user associated with the searchdevice 106, and/or viewership or number of downloads for the individualitems.

Alternatively, the search system 108 can provide an item identified inthe search results 112 to the search device 106. For example, the searchsystem 108 can receive the web page or Internet video from the networkapplication data provider 114 and forward the item to the search device106.

In some implementations, the search system 108 receives and storesadditional content available from computing devices on the network 118prior to receiving the search query 110. This can remove an additionaltrip over the network 118 (to the network application data provider 114)at the time when the additional content is provided to the search device106.

The network application data provider 114 is a computing system, such asa web server or other network accessible computing system. The networkapplication data provider 114 stores one or more web pages, video,audio, gadgets, or applications including one or more of the itemslisted in the search results 112. The network application data provider114 can send a network application data 116 over the network 118 to thesearch device 106. For example, the network application data provider114 can send the network application data 116 to be processed by agadget or application presented by the display device 102.

In addition to sending the search query 110 to the search system 108,the search device 106 can search data stored locally at the searchdevice 106 using the search query 110. For example, the search device106 can search within web pages, gadgets, applications, video, or filesthat have already been downloaded and/or installed at the search device106. The local search results can include links for opening orinitiating the existing items. The search device 106 can also searchuser data at the search device 106 for additional content that satisfiesthe search query 110. For example, the search device 106 may includebookmarks or shortcut links to web pages or other network accessibledata and the search device 106 can search the bookmarks and/or shortcutsfor web pages or data matching the search query 110. The search device106 and/or the search system 108 can store a browsing history or searchquery history which can be searched using the search query 110. Thesearch device 106 can, in some implementations, display local searchresults immediately and displays search results from the search system108 as they arrive at the search device 106.

In addition to sending the search query 110 to the search system 108 andsearching data that is at the search device 106, additional content onthe set-top boxes 105 a-b can be searched. For example, one or more ofthe set-top boxes 105 a-b can include a DVR and the search device 106can have access to a listing of television programming and/or othercontent stored on the set-top boxes 105 a-b. The search device 106 canthen use the search query 110 to search through the recorded televisionprogramming and/or other stored content for additional content fordisplay on the display device 102. Alternatively, the search device 106can send the search query 110 to the set-top boxes 105 a-b. The set-topboxes can then perform the search and return the search results to thesearch device 106. In another example, one or more of the set-top boxes105 a-b can report a listing of television programming and other contentstored at the respective set-top box to an external server system. Thesearch device 106 can then search the external server system or providethe search query to the external server system.

The search device 106 can generates new search queries when thetelevision programming shown on the television programming displaydevice 102 changes or when a request for searching or a search query isreceived from the user. For example, when the channel is changed, adifferent television program is presented, or when a change occurswithin a television program, the search device 106 can generate a newsearch query. In some implementations, the search device 106automatically generates a search query when the search device 106determines that the metadata for the television programming includes anexplicit reference to additional content. For example, the transcript ofthe television programming, recognized text from audio, or recognizedtext from video for the television programming may indicate that anaddress of a web page was presented. The search device 106 can initiatea search for the address and present the web page or search results forthe identified address on the display device. In another example, thesearch device 106 can determine that the metadata includes a referenceto another television program or television channel. Accordingly, thesearch device 106 can initiate a search for the identifiedprogram/channel and present a list of results on the display device.

In another implementation, some processing can be performed by thesearch system 108. For example, the metadata can be sent to the searchsystem 108 or another external system. The search system 108 can thenanalyze the metadata to determine search query suggestions and/orperform one or more searches using the search suggestions to identifysearch results. The search system 108 can then send the searchsuggestions and/or search results to the search device 106 for displayon the display device 102.

In one example, a user may be watching an instructional cookingtelevision program called “Cooking with Sparky.” In this example, thetelevision programming provider 104 is a local public broadcastingstation. The television programming display device 102 in this exampleis a television that receives the television programming from the searchdevice 106 through an over-the-air transmission. The search device 106records and stores information about the channel that is currentlyselected by and displayed to the user.

In some implementations, the search device 106 receives informationabout the specific television program that is being displayed, such as atitle of the television program. For example, the search device 106 canreceive electronic program guide (EPG) information from a server systemover the network 118. In another example, the search device 106 canreceive closed captioning (CC) information or other information withinthe signal or separately from the signal from the television programmingprovider 104 or another system. In some implementations, the EPG and CCinformation is retrieved from metadata in a broadcast signal, such asthe metadata in the Vertical Blanking Interval or the metadata providedby the Program and System Information Protocol (PSIP). EPG metadata caninclude, for example, a title of the television program, a descriptionof the television program, the channel name and/or number, and the timeinterval and date for the broadcast of the television program. CCmetadata can include, for example, a machine generated or manualgenerated transcript of the television program.

In some implementations, the search device 106 can retrieve metadata forthe current television program from an external source. For example, thesearch device 106 can use the identity of the channel and the currenttime and date to uniquely identify the television program. The searchdevice 106 can then request the EPG and/or CC data from an externalsource, such as the search system 108 or another computing system on thenetwork 118. Alternatively, the search device 106 can download EPGand/or CC metadata for a number of channels and television programsprior to the user selecting a channel. The search device 106 can thensearch for the television program in the local copy of the EPG and/or CCmetadata rather than requesting the metadata at the time the user makesa selection.

Returning to the cooking show example, based on the time, date andprogramming information previously downloaded, the search device 106 canidentify that the show being displayed is “Cooking with Sparky.” Thesearch device 106 can generate the search query 110, including termsapplicable to this television program. For example, the search query 110can include EPG information that describes the television program, suchas information identifying the channel as a public broadcasting station,identifying the show as one that often features barbeque cooking, andthat the show is hosted by a former stunt man named Sparky. In someimplementations, the search query 110 can also include information aboutthe user of the television programming display device 102, the hardwareused to output the video, and/or other information. For example, if theviewer indicates a preference for Spanish language translations, thisinformation can be included in the search query 110.

The search device 106 sends the search query 110 over the network 118 tothe search system 108. The search system 108 performs the search query110 on an index of web pages and other data. The search results 112include a list of items related to the television program “Cooking withSparky” and to properties of the user or the user's system. The searchsystem 108 sends the search results 112 to the search device 106.

In one example, the search results 112 include a gadget or applicationthat scans closed caption text for cooking measurements and displaystheir conversion values (such as from metric to standard units). Inanother example, the search results 112 include an advertisement for afund raiser held by the public broadcasting station. The search results112 may also include a gadget that shows Sparky's entry in an onlineencyclopedia of movies that lists the movies in which Sparky hasparticipated as a stunt man. In another example, the search results 112include a translator that displays the closed captioning text translatedinto another language.

In some implementations, the search results 112 are sent to the networkapplication data provider 114. The network application data provider 114can then send an item (e.g., a web page, gadget, or application) fromthe search results 112 to the search device 106. The search device 106then processes the item and overlays the item on the audio/video signalpassed to the television programming display device 102.

In some implementations, the television programming display device 102can be a personal computer. For example, the television programmingprovider 104 can be an Internet website operated by a nationalbroadcasting company that streams episodes of television shows. In thisexample, the search device 106 can be implemented as software and/orhardware in the personal computer. The viewer may be watching lastweek's episode of “Who Knows how to Cook?,” which is a realitytelevision show where contestants battle in a culinary tournament to wina restaurant. The search device 106 generates the search query 110 basedon metadata about the show. The search query 110 includes informationidentifying the show as a reality television show, that the currentepisode features barbeque cooking, and that the show is hosted by famousrestaurateur Chef Emanuel. The search device 106 sends the search query110 to the search system 108. The search system 108 processes the searchquery 110 to identify a list of additional content having informationthat matches the search terms in the search query 110 (e.g., web pages,gadgets, and/or applications related to reality television, barbequecooking, and/or Chef Emanuel).

For example, the search system 108 can identify a chat program able toconnect the user with a chat room filled other people currently watching“Who Knows how to Cook?” The search system 108 can identify the gadgetthat scans closed captioning text for cooking measurements and displaystheir converted values. The search system 108 can identify a web page ofa restaurant near the user that is owned and operated by Chef Emanuel.The search system 108 sends the list of additional content to the searchdevice 106 in the search results 112. In some implementations, thesearch system 108 sends links to the additional content, such as aUniform Resource Identifier (URI). The search device 106 can use the URIinformation in the search results 112 to request the additional contentfrom the network application data providers 114 that host one or more ofthe items in the search results 112. In some implementations, the searchsystem 108 sends the additional content without requiring the searchdevice 106 to send a request to retrieve the additional content.

FIG. 2 is a schematic diagram that shows an example of a system 200 fortelevision related searching. The system 200 includes a search device202 in wire line or wireless communication with a television 204, apersonal computer 206, and a wireless video player 208. The television204, the personal computer 206, and the wireless video player 208present audio/video data received from the search device 202, includingadditional content output from a gadget, web page, or application thatthe search device 202 can overlay on the audio/video data. Other systemsincluding more or fewer video playback devices and similar or differenttypes are also possible. The search device 202 is also in communicationwith a search system 210 and one or more network application dataproviders 216 a-d over a network 222, such as the Internet.

The search system 210 includes a search engine 212 that processes searchqueries generated by the search device 202. For example, the searchengine 212, or another module or server, may crawl the network 222 toidentify content that is available from the network application dataproviders 216 a-d over the network 222. Alternatively, the networkapplication data providers 216 a-d or other systems can provideinformation about available content, such as links to the networkapplication data providers 216 a-d. In another example, the networkapplication data providers 216 a-d can upload content, such as webpages, video, audio, gadgets, or applications to the search system 210for download by the search device 202 or the search device 202 canretrieve content directly from the network application data providers216 a-d. In some implementations, the search system 210 provides an appstore where applications can be purchased and then downloaded for use bya device, such as the search device 202.

The search system 210 stores information for the content in an index214. The index 214 can include information such as a name, keywords, atelevision channel number, a television channel name, a televisionprogram name, a time of day, a network address, or other informationassociated with the content. In some implementations, the index 214includes a copy of the content allowing the search system 210 to sendthe content directly to the search device 202.

The network application data providers 216 a-d are computing systemsthat provide access to content over the network 222. The networkapplication data providers 216 a-d include respective web servers 218a-d and one or more web pages 220 a-d. The web servers 218 a-d processincoming and outgoing communications. The web servers 218 a-d provideinteractivity with the web pages 220 a-d. The web pages 220 a-d can beshown, for example, in a gadget. While some of the web pages 220 a-d maybe in a format able to be presented directly by a web browser, such asHTML, some image formats, and JavaScript, other data types may bepresented within the browser using plug-ins, such as Java or Flashmedia.

In one example, the television 204 connected to the search device 202shows the “Cooking with Sparky” television program. The search device202 can use information from the search system 210 to identify atranslation gadget from the network application data provider 216 b. Thesearch device 202 communicates with the web server 218 b to request thetranslation gadget from the web pages 220 b.

In another example, the personal computer 206 (e.g., in another room ofthe same house as the television 204) shows the “Who Knows how to Cook?”television program. The search device 202 can use information from thesearch system 210 to identify a fan website from the network applicationdata provider 216 d. The search device 202 communicates with the webserver 218 d to request the fan page for the “Who Knows how to Cook?”television program.

FIG. 3 is a block diagram that shows an example of a search device 300for television related searching. The search device 300 accepts inputs,such as a television signal 302 through an interface 310 and a userchannel request 306 through an interface 307. For example, the interface310 can be a hardware interface such as a consumer electronics videoconnection. The interface 307 can be a hardware interface, such as awireless infrared receiver for a remote control, a wire line networkconnection to a computing device, or a button on the search device 300itself.

The search device 300 can receive an audio/video signal for a televisionchannel that has already been tuned by a tuner that is external to thesearch device 300. For example, the search device 300 can receive theuser channel request 306 and forward a channel request 315 on to theexternal tuner through an interface 309. The interface 309 may be aconsumer electronics connection, such as an HDMI connection or an IRconnection. The search device 300 then receives the television signal302 from the external tuner for the channel that was requested in thechannel request 315. In some implementations, the interface 310 and theinterface 309 can be colocated or a single interface, such as in an HDMIconnection.

As an alternative to an external tuner, the search device 300 mayinclude a tuner. The search device 300 uses the tuner and the userchannel request 306 to tune in an audio and video signal 311 for aparticular television channel. For example the search device 300 caninclude an Advanced Television Systems Committee (ATSC) tuner, aNational Television Systems Committee (NTSC) tuner, a Sequential Colorwith Memory (SECAM) tuner, a Phase Alternating Line (PAL) tuner, and/ora Digital Video Broadcasting (DVB) tuner. The tuner can allow the searchdevice 300 to tune one or more of over the air broadcast televisionchannels, cable television channels, satellite television channels, orchannels provided by a telecommunications/telephone company.

In addition to live broadcast by an internal or external tuner, thesearch device 300 can receive and/or retrieve recorded televisionprogramming or video on demand. In addition to video programming, thesearch device 300 can receive audio channels or radio as the inputsignal.

The search device 300 includes a metadata decoder 312 that decodesmetadata from the television signal 302. For example, the metadatadecoder 312 can decode certain metadata 313 from the television signal302, such as call letters for the television channel, the name of thetelevision program, closed captioning text of the television program, orother metadata included in the television signal 302. The metadatadecoder 312 provides the audio and video signal 311 for the televisionprogram and the metadata 313 for the television program to a searchmanager 314. In some implementations, the search manager 314 receives inband metadata if the interface 309 and/or the interface 310 are analoginterfaces.

In some implementations, the metadata 313 provided to the search manager314 is general purpose information that is not specifically designed tobe a trigger for additional content. For example, this may allow thesearch manager 314 to be used with general purpose televisionprogramming systems rather than a special purpose television programmingprovider system specifically designed for presenting additional content.

In some implementations, the search manager 314 can receive out of bandmetadata through an interface 316. For example, the interface 316 can bea network connection to a local area network or the Internet. The searchmanager 314 can use, for example, a channel identifier, a time of dayand date, and television programming provider head end information(e.g., a name of the provider and/or a location of the search device300) to access or retrieve EPG data, transcript data, or other out ofband programming data related to the television signal 302. In someimplementations, the search manager 314 receives out of band metadatathrough the interface 316 if the interface 309 and/or the interface 310are digital interfaces.

The search manager 314 generates a search query 318 based on themetadata 313 and/or any out of band metadata. The search manager 314sends the search query 318 to a remote computing system through theinterface 316. In some implementations, the interface 316 can be anetwork interface to the Internet, an intranet, or other computernetwork.

The remote computing system processes the search query 318 to identifyone or more search results 320. The remote computing system sends thesearch results 320 to the search manager 314. The search results 320 caninclude links to additional content, such as a web page, a videoavailable on a web site, a product for sale at an Internet retailer website, a television program, a television channel, or an application. Theitems listed in the search results 320 may include items available atthe search device 300 or items from a remote system. For example, thesearch results 320 can include a link to an application that isinstalled at the search system 300 or a link to an application that canbe downloaded from a remote system.

In some implementations, the search manager 314 stores items downloadedfrom links in the search results 320 in a data storage 322. In someimplementations, the data storage 322 is a persistent memory, such as adatabase in a hard drive or flash memory device. In someimplementations, the data storage 322 is a non-persistent memory, suchas random access memory (RAM). The search manager 314 can also use thesearch query 318 to search the data storage 322 for items that havealready been downloaded and/or installed at the search device 300.

In some implementations, where the search results 320 include multiplecontent items, the search manager 314 presents a list of the items tothe user and allows the user to select one or more of the items forpresentation, download, or recording. In some implementations, thesearch manager 314 selects a predetermined number of the items resultingfrom the search (such as the first one, two, or three items) for displayand provides an option that allows the user to request that one or moreof the non-displayed items be displayed. In some implementations, thesearch manager 314 selects all of the items resulting from the searchfor display on the television device. In another example, the searchmanager 314 selects enough items to fill the space available on thedisplay device and can also provide a user input control to request thedisplay of additional items. In another example, the search manager 314can hide or disable a gadget or application that has not been used orhas been inactive for a threshold amount of time to provide space forother content.

The search manager 314 combines the audio and video signal 311 withcontent from one or more of the search results 320. For example, thesearch manager 314 can overlay the audio and visual data from thecontent on top of the corresponding components of the audio and videosignal 311 to produce an audio and video signal with content overlay304. The search manager 314 outputs the audio and video signal withcontent overlay 304 through an interface 324 to a device such as thetelevision programming display device 102, the television 204, thepersonal computer 206, or the wireless video player 208. In someimplementations, the interface 324 includes one or more consumerelectronics audio/video hardware interfaces, such as High-DefinitionMultimedia Interface (HDMI), Digital Visual Interface (DVI), componentvideo, composite video, Video Graphics Array (VGA) D-subminiature(DSUB), S/PDIF digital audio (e.g., over coaxial cable or optical fiber,such as TOSLINK), and/or analog audio.

In some implementations, the search manager 314 outputs the audio andvideo signal with content overlay 304 over a network interface, such asthe interface 316. In addition, the search manager 314 can convert theaudio and video signal with content overlay 304 to a compressed format,such as the MPEG-2 format or the H.264 format. The search manager 314can then send the compressed signal to a computing device on the localarea network with the search manager 314 or to a remote computingdevice, such as a device connected to the search manager 314 through theInternet.

Over time the metadata decoder 312 sends updates of the metadata 313 tothe search manager 314. For example, if the user inputs a new channelrequest, the metadata decoder 312 or the interface 307, can pass the newchannel identifier to the search manager 314.

In another example, if metadata within the television signal 302indicates that a previous television program has ended and anothertelevision has begun, then the metadata decoder 312 can send thetelevision program status updates to the search manager 314. In anotherexample, the search manager 314 can determine that a television programhas ended and a new television program has begun based on the currentchannel identifier in the metadata 313 and EPG metadata. In someimplementations, the search manager 314 retrieves the EPG metadata atruntime from an external source. In some implementations, the searchmanager 314 retrieves the EPG metadata at some time prior to the searchand stores the EPG metadata in a storage device, such as the datastorage 322.

In another example, the metadata decoder 312 can send continuousmetadata updates to the search manager 314, such as in the case ofclosed captioning text. As the metadata decoder 312 decodes the closedcaptioning text from the television signal 302, the metadata decoder 312sends the closed captioning text to the search manager 314.

The search manager 314 uses the metadata 313 to manage the contentcurrently being displayed in the audio and video signal with contentoverlay 304. For example, if the user changes the channel, the channelchange is reflected in the metadata 313 and the search manager 314 canperform a new search query using the new channel and EPG informationassociated with the current time and the new channel. In one example, ifthe search manager 314 is currently displaying a gadget that is nolonger included in the new search results, then the search manager 314can remove the gadget from the audio and video signal with contentoverlay 304. In some implementations, the search manager 314 prompts theuser to accept the removal of the gadget that no longer satisfies thesearch query.

In some implementations, the search manager 314 publishes data to whichthe gadgets or applications can subscribe. For example, the searchmanager 314 can publish the metadata 313 from the metadata decoder 312or the out of band metadata. In another example, the search manager 314can publish text or objects recognized from the video in the televisionsignal 302 and/or text recognized from the audio in the televisionsignal 302. In some implementations, the text and/or object recognitionof the video or audio can be performed by a remote system. In anotherimplementation, the search manager 314 can perform text and/or objectrecognition of the video or audio.

A gadget or application can subscribe to a stream including thepublished data. The gadget or application can then use the publisheddata, such as in translating text into another language. In anotherexample, the search manager 314 can publish other television programmingrelated information, such as an identifier of the channel or televisionprogram being presented.

In some implementations, the search manager 314 can publish informationrelated to the user, such as a location of the user (e.g., a city orpostal code), television program viewing preferences of the user, orother user information. In some implementations, the search manager 314only publishes user information upon request by the user. For example,the search manager 314 can prompt the user for confirmation each time agadget or application requests subscription to user information or otherinformation. In another example, the search manager 314 can apply aconfirmation to all gadgets or applications or a group of gadgets and/orapplications, such as a group from the same Internet domain or from thesame author.

FIG. 4 is an example of a graphical user interface (GUI) 400 fortelevision related searching. The GUI 400 presents one or more searchsuggestions 402 a-c and one or more search results 404 a-g. The searchsuggestions 402 a-c and the search results 404 a-g are presented in awindow 406 that is overlaid on video for a television program 408. Thewindow includes a search input control 410. A user has entered the text“nb” into the search input control 410. A search device, such as thesearch device 300, processes the search term “nb” and generates thesearch suggestions 402 a-c. The search device, or a search system incommunication with the search device, determines that the user mayintend to search for “nb” itself, “nbc,” or “nba” and provides these asthe search suggestions 402 a-c, respectively. The suggestions may bebased on, for example, the user's previous searches or popular searchesfrom other users. In addition, the suggestions can include synonyms orterms frequently used in conjunction with the entered term. The searchresults 404 a-g include items that match the query for the searchsuggestion 402 a.

The search results 404 a-g include results of performing a search usingone or more of the search suggestions 402 a-c. For example, the searchmay be performed using the first suggestion or the suggestion mostfrequently used by the user or other users. Alternatively, the searchmay be performed using all of the suggestions or a predetermined numberof the suggestions, such as the top two. Where multiple suggestions areused, the corresponding results can be presented separately orintermingled. For intermingled results, the individual results can beprocessed to remove duplicates and ordered with the more relevantresults be listed in a higher position than less relevant results.Relevancy can be based on the number of times the search terms appear inthe content for the item and/or the number of times the user or otheruser have selected the item. Relevancy can also be based on thepopularity of the television channel, the popularity of the televisionprogram, the quality of the video (e.g., high definition may be rankedhigher than standard definition or vice versa), how old an episode of atelevision program is, or how soon an episode of a television programwill be broadcast. For example, the next broadcast of a televisionprogram may be ranked higher than subsequent broadcasts or a currentlybroadcast episode of a television program may be ranked higher thanrecorded episodes or episodes to be broadcast in the future.

The search results can include multiple types of content. For example,the search result 404 e includes a television channel. The search result404 e identifies the name of the channel as “NBC.” A user can select thesearch result 404 e to request that the search device tune to thatchannel. The search results 404 a and 404 c also include televisionprograms. The television program search results include informationidentifying the name or title of the program as well as the day and timethat the program is or will be broadcast. If a television program listedin the search results is currently being broadcast, a user can selectthe search result to request that the search device tune to the channelfor the search result. If a television program listed in the searchresults is broadcast in the future, a user can select the search resultto request that the search device record the television program. Thechannel change or recording operation may be preceded by a prompt fromthe search device for the user to confirm the operation. In someimplementations, the search device sends the channel change and/orrecord command to an external device, such as a DVR or programmingprovider set-top box.

The search result 404 f includes an application. The applicationprovides score and updates for the NBA. If the application is alreadyinstalled at the search device, the user can select the search result404 f to open the application. Alternatively, if the application is notyet installed at the search device, then the search device can provide aprompt to the user for confirming installation of the applications. Insome implementations, the search results only include applicationresults for applications that are already installed at the searchdevice. Once selected, the user interface for the application ispresented on the display device with the television programming. In someimplementations, the window 406 for the search results is automaticallyclosed upon the user's selection of the application in the searchresults.

The search results 404 b, 404 d, and 404 g include web pages. The webpage search results indicate a title or name for each of the web pagesas well as a URL for each of the web pages. A user can select a web pagesearch result to navigate to the selected web page. In someimplementations, the window 406 is a browser and the selected web pageis displayed in the window 406. The search device can update the searchinput control 410 to replace the search text with the URL of thedisplayed web page.

The window 406 includes one or more controls 412 for changing how theadditional content is displayed. For example, a user can select acontrol to split the television programming 408 and the window 406 intoleft and right halves of the GUI 400. In another example, the GUI 400can be divided into a larger number of portions, such as four quarters.The television programming 408 can be placed in one quarter while thewindow 406 can be placed in a second quarter and other content such asweb pages or applications can be placed in the other portions of the GUI400. A user can also select a control to stack or tile the window 406and windows for other web pages or applications. In another example, thetelevision programming can appear in a thumbnail or overlay while thewindow 406 or other additional content appears in full screen.

In some implementations, presentation of the search suggestions can beinitiated by a request for suggestions from the user rather than asearch input from the user. The search device can provide thesuggestions based on metadata for the television programming currentlybeing displayed. For example, a user may select an input controlrequesting additional content for the television programming currentlydisplayed. The search device can use the metadata to generate searchsuggestions and/or search results. The search device then presents thesearch suggestions and/or search results in the GUI 400.

In another implementation, presentation of the search suggestions can betriggered by information in the metadata for the television programmingbeing presented to the user. For example, if a television program orcommercial mentions or displays a URL for a web page or other dataavailable at a URL, then the search device can initiate a search and/orsuggestions for the URL. In some implementations, the suggestions and/orresults can be displayed briefly and be removed automatically if theuser does not select a suggestion or search result within a certainamount of time. In addition to URLs, the metadata can triggersuggestions or results using other identifying information. For example,a name of a television program or a name or call letters of a televisionchannel can trigger suggestions or search results. A user can thenselect the television program and/or channel result to display or recordthat program/channel.

FIG. 5 is a flow chart that shows an example of a process 500 fortelevision related searching. The processes 500 can be performed, forexample, by a system such as the system 100, the system 200, and thesearch device 300. For clarity of presentation, the description thatfollows uses the system 100, the system 200, and the search device 300as examples for describing the process 500. However, another system, orcombination of systems, may be used to perform the process 500.

The process 500 receives (502), by for example the search device 202, auser request to perform a search related to television programming beingpresented to the user. In some examples, this can include a userpressing a button on a remote control, clicking an on screen button, orturning to a new television program. In some implementations, this stepcan be optional and/or skipped. In some examples, this includesperforming the following steps at predetermined time intervals, when achange in television programming is detected, and/or continuously.

The process 500 receives (504), by for example the search system 210,metadata related to the television programming being presented to theuser. In some examples, this includes the metadata 304 being received bythe metadata decoder 310.

The process 500 extracts (506), by for example the search manager 314,multiple keywords from the metadata. In some examples, this includesidentifying an initial list of keywords, which can generate additionalkeywords to produce search terms that can be more readable. The keywordscan be presented to a user as search suggestions.

The process 500 receives (508), by for example the search manager 314, auser selection of at least one of the keywords. In some examples, thisincludes a user selecting an individual search term or a set of terms ina phrase.

The process 500 performs (510), by for example the search system 210and/or the search device 202, a search, using the at least one selectedkeyword, for additional content related to the television programmingbeing presented to the user. In some examples, this includes searchingfor the search terms in an index of web pages and other contentavailable on the network 222 and ranking the search results based onrelevance, date of creation, or other factors.

The process 500 provides (512), by for example the search device 202,results of the search to a client device for presentation to the user.In some examples, this includes overlaying the search results with thetelevision programming being displayed. In other examples, thetelevision programming being displayed can be suspended or reduced toshow the search results.

FIG. 6 is a schematic diagram that shows an example of a computingsystem 600. The computing system 600 can be used for the operationsdescribed in association with any of the computer-implement methods andsystems described previously, according to some implementations. Thecomputing system 600 includes a processor 610, a memory 620, a storagedevice 630, and an input/output device 640. Each of the processor 610,the memory 620, the storage device 630, and the input/output device 640are interconnected using a system bus 650. The processor 610 is capableof processing instructions for execution within the computing system600. In some implementations, the processor 610 is a single-threadedprocessor. In some implementations, the processor 610 is amulti-threaded processor. The processor 610 is capable of processinginstructions stored in the memory 620 or on the storage device 630 todisplay graphical information for a user interface on the input/outputdevice 640.

The memory 620 stores information within the computing system 600. Insome implementations, the memory 620 is a computer-readable medium. Insome implementations, the memory 620 is a volatile memory unit. In someimplementations, the memory 620 is a non-volatile memory unit.

The storage device 630 is capable of providing mass storage for thecomputing system 600. In some implementations, the storage device 630 isa computer-readable medium. In various different implementations, thestorage device 630 may be a floppy disk device, a hard disk device, anoptical disk device, or a tape device.

The input/output device 640 provides input/output operations for thecomputing system 600. In some implementations, the input/output device640 includes a keyboard and/or pointing device. In some implementations,the input/output device 640 includes a display unit for displayinggraphical user interfaces.

The features described can be implemented in digital electroniccircuitry, or in computer hardware, firmware, software, or incombinations of them. The apparatus can be implemented in a computerprogram product tangibly embodied in an information carrier, e.g., in amachine-readable storage device or in a propagated signal, for executionby a programmable processor; and method steps can be performed by aprogrammable processor executing a program of instructions to performfunctions of the described implementations by operating on input dataand generating output. The described features can be implementedadvantageously in one or more computer programs that are executable on aprogrammable system including at least one programmable processorcoupled to receive data and instructions from, and to transmit data andinstructions to, a data storage system, at least one input device, andat least one output device. A computer program is a set of instructionsthat can be used, directly or indirectly, in a computer to perform acertain activity or bring about a certain result. A computer program canbe written in any form of programming language, including compiled orinterpreted languages, and it can be deployed in any form, including asa stand-alone program or as a module, component, subroutine, or otherunit suitable for use in a computing environment.

Suitable processors for the execution of a program of instructionsinclude, by way of example, both general and special purposemicroprocessors, and the sole processor or one of multiple processors ofany kind of computer. Generally, a processor will receive instructionsand data from a read-only memory or a random access memory or both. Theessential elements of a computer are a processor for executinginstructions and one or more memories for storing instructions and data.Generally, a computer will also include, or be operatively coupled tocommunicate with, one or more mass storage devices for storing datafiles; such devices include magnetic disks, such as internal hard disksand removable disks; magneto-optical disks; and optical disks. Storagedevices suitable for tangibly embodying computer program instructionsand data include all forms of non-volatile memory, including by way ofexample semiconductor memory devices, such as EPROM (erasableprogrammable read-only memory), EEPROM (electrically erasableprogrammable read-only memory), and flash memory devices; magnetic diskssuch as internal hard disks and removable disks; magneto-optical disks;and CD-ROM (compact disc read-only memory) and DVD-ROM (digitalversatile disc read-only memory) disks. The processor and the memory canbe supplemented by, or incorporated in, ASICs (application-specificintegrated circuits).

To provide for interaction with a user, the features can be implementedon a computer having a display device such as a CRT (cathode ray tube)or LCD (liquid crystal display) monitor for displaying information tothe user and a keyboard and a pointing device such as a mouse or atrackball by which the user can provide input to the computer.

The features can be implemented in a computer system that includes aback-end component, such as a data server, or that includes a middlewarecomponent, such as an application server or an Internet server, or thatincludes a front-end component, such as a client computer having agraphical user interface or an Internet browser, or any combination ofthem. The components of the system can be connected by any form ormedium of digital data communication such as a communication network.Examples of communication networks include, e.g., a LAN (local areanetwork), a WAN (wide area network), and the computers and networksforming the Internet.

The computer system can include clients and servers. A client and serverare generally remote from each other and typically interact through anetwork, such as the described one. The relationship of client andserver arises by virtue of computer programs running on the respectivecomputers and having a client-server relationship to each other.

Although a few implementations have been described in detail above,other modifications are possible. In addition, the logic flows depictedin the figures do not require the particular order shown, or sequentialorder, to achieve desirable results. In addition, other steps may beprovided, or steps may be eliminated, from the described flows, andother components may be added to, or removed from, the describedsystems. Accordingly, other implementations are within the scope of thefollowing claims.

What is claimed is:
 1. A computer-implemented method for televisionrelated searching, the method comprising: causing a media program to bepresented on a display device; identifying metadata related to the mediaprogram being presented on the display device; extracting a plurality ofkeywords from the identified metadata; in response to extracting theplurality of keywords from the identified metadata, automaticallygenerating a plurality of search suggestions based on the plurality ofextracted keywords and based on search results responsive to each of theplurality of search suggestions, wherein additional search suggestionsare generated in the plurality of search suggestions in response todetermining that a television programming change has occurred; obtaininga plurality of search results responsive to at least a portion of theplurality of search suggestions, wherein each of the plurality of searchresults is associated with a content type of a plurality of contenttypes, and wherein a first search result of the plurality of searchresults is associated with a television channel content type andincludes a first identifier corresponding to a television program thatis scheduled to be broadcast at a future time, a second search result ofthe plurality of search results is associated with an applicationcontent type and includes a second identifier corresponding to anapplication for execution on the display device, and a third searchresult of the plurality of search results is associated with a web pagecontent type and includes a third identifier corresponding to webcontent for presentation on the display device; causing a first portionof the plurality of search suggestions and a second portion of theplurality of search results that includes the first search result, thesecond search result, and the third search result to be presentedtogether on the display device in an overlay that is positioned over themedia program in response to being triggered by information in themetadata, wherein: the first search result includes the first identifierthat, in response to receiving a selection of the first identifier,causes the television program that is scheduled to be broadcast at thefuture time to be recorded at the future time; the second search resultincludes the second identifier that, in response to receiving aselection of the second identifier, determines whether the applicationhas been installed, causes a prompt to install the application to bepresented in response to determining that the application has not beeninstalled, and causes the overlay to be removed and the application tobe launched in response to determining that the application has beeninstalled; the third search result includes the third identifier that,in response to receiving a selection of the third identifier, causes theweb page content corresponding to the third search result to bepresented within the overlay in place of the first portion of theplurality of search suggestions and the second portion of the pluralityof search results that includes the first search result, the secondsearch result, and the third search result; and the overlay isautomatically removed in response to a time period elapsing in which oneof the first portion of the plurality of search suggestions and thesecond portion of the plurality of search results has not been selected;receiving a user selection of the first identifier corresponding to thefirst search result of the plurality of search results; and in responseto receiving the user selection of the first identifier corresponding tothe first search result, causing content corresponding to the selectedfirst identifier to be recorded at the future time by transmitting arecord command to a recording device associated with the display device.2. The method of claim 1, wherein the television programming changeincludes a change from a first channel to a second channel.
 3. Themethod of claim 1, wherein the television programming change includes achange from the media program to a different media program.
 4. Themethod of claim 1, wherein the television programming change includes achange within the media program.
 5. The method of claim 1, whereinobtaining the plurality of search results further comprises performing asearch that inputs each of a predetermined number of the plurality ofsearch suggestions to a search engine system and wherein the pluralityof obtained search results are ranked in the overlay based on relevancy.